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ABSTRACT 

We  have  investigated  side  effects  and  aftereffects  evoked  by  moving  the  head  to  interact  with  a virtual  environment  (VE) 
shown  in  a helmet  mounted  visual  display  (HMD).  The  graphics  computer  of  such  a VE  must  monitor  the  HMD’s 
spatial  orientation  and  position  in  order  to  present  images  from  the  proper  perspective.  Delays  between  head  movements 
and  image  updating  cause  aberrant  visual  motion  of  a virtual  world.  We  found  that  above  delays  of  40  ms  motion 
sickness  and  postural  instability  are  evoked  minutes  after  head  movements  begin.  The  severity  of  side  effects  is  a 
function  of  the  latency  between  head  movement  and  visual  update  delay.  Fifteen  minutes  of  making  head  movements  in 
a VE  with  a 254  ms  delay  causes  motion  sickness  severe  enough  to  make  28%  of  subjects  withdraw  from  the  situation. 
Users  appear  recovered  1 5 minutes  after  VE  exposure  ends  if  they  remain  immobile,  but  normal  activities  quickly  revive 
their  motion  sickness  symptoms,  indicating  that  they  were  sensitized  by  exposure  to  visual  update  delays.  We  conclude 
that  visual  update  delays  are  a unique  cause  of  side  effects  and  aftereffects  in  VEs  utilizing  HMDs. 

INTRODUCTION 

Virtual  environment  (VE)  technology  has  many  potential  applications  but  current  technology  leads  to  side  effects  and 
aftereffects  for  users.  Motion  sickness  is  an  acknowledged  problem  in  VEs  (DiZio  & Lackner,  1992,  1997;  Wilson, 
1996)  that  needs  to  be  solved  because  it  is  aversive  and  hinders  performance  of  normal  activities  after  the  user  leaves  the 
VE. 

We  have  attempted  to  document  the  incidence  and  etiological  factors  of  motion  sickness  side  effects  and  after  effects 
associated  with  a prevalent  VE  configuration  which  utilizes  an  HMD.  For  experimental  investigation,  we  developed  a 
facsimile  of  the  VESUB  system  developed  at  NAWC-TSD  in  Orlando,  Florida  for  training  naval  candidates  to  navigate  a 
surfaced  submarine  in  a harbor.  Subjects  view  a virtual  harbor  scene  from  the  perspective  of  a surfaced  submarine.  One 
sub-goal  of  both  VESUB  and  of  real  officer  of  the  deck  training,  is  for  the  user  to  develop  spatial  awareness  by  scanning 
the  environment  for  signs  and  landmarks  spread  out  over  360°.  Therefore,  the  VESUB  system  and  our  experimental 
system  utilize  HMDs,  which  enable  presentation  of  a panoramic  view  of  the  virtual  world  from  a perspective  that  can  be 
adjusted  by  glancing  about  in  a natural  manner. 

The  scenery  of  a virtual  world  in  an  HMD  is  carried  around  by  the  head  instead  of  staying  fixed  like  the  real  scenery.  In 
such  VEs,  head  position  is  monitored  by  a tracking  device  and  fed  to  a graphics  computer  which  updates  the  user’s 
perspective  and  computes  the  proper  image  for  the  HMD  display  units,  This  gives  the  appearance  of  a stable  virtual 
world  and  appears  to  contribute  to  the  sense  of  immersion  in  the  VE  (Deisinger,  Cruz-Neira,  Riedel,  & Symanzik,  1997). 
However,  the  process  is  imperfect  in  practice  because  of  delays  in  the  tracking  device,  the  transmission  of  tracker  signals, 
graphical  computation  and  rendering  time  and  others.  These  delays  cause  aberrant  slippage  of  the  retinal  image  when  a 
head  movement  is  made.  That  is,  the  HMD  images  commence  moving  later  than  the  head  and  continue  for  a latent  period 
after  the  head  movement  is  complete. 

In  flight  simulators  utilizing  large  screen  displays,  motion  sickness  and  postural  instability  are  prevalent  side-  or 
aftereffects,  but  head  movements  per  se  have  never  been  identified  as  etiological  factors  (Kennedy,  Hettinger, 
Lilienthal,  1990).  In  our  VE,  the  first  and  most  obvious  thing  we  noticed  was  that  users  rapidly  became  motion  sick 
when  they  looked  about  wearing  the  HMD  but  did  not  complain  if  they  kept  still  while  viewing  a stationary  virtual 
world.  The  only  visual  motion  in  our  VE  was  brought  about  by  delayed  updating  of  the  visual  scene  in  the  HMD  during 
head  movements.  Thus,  we  launched  an  effort  to  determine  whether  visual  update  delays  associated  with  head 
movements  are  a specific  cause  of  side-effects  and  aftereffects  in  VEs  utilizing  HMDs. 


Paper  presented  at  the  RTO  HFM  Workshop  on  "The  Capability  of  Virtual  Reality  to  Meet  Military 
Requirements”,  held  in  Orlando,  USA,  5-9  December  1997,  and  published  in  RTO  MP-54. 
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VE  SYSTEM  CHARACTERISTICS 

VE  computer  and  software : The  virtual  environment  we  experimented  with  was  generated  with  a Silicon  Graphics 
ONYX  Reality  Engine  II  computer  (SGI).  The  virtual  world  was  a harbor  scene  from  the  perspective  of  the  sail  of  a 
surfaced  submarine.  The  boat  was  still  in  the  water,  and  there  was  no  wave  motion  or  any  other  external  visual  motion. 
The  harbor  channel  was  marked  by  buoys  in  the  water  and  range  markers  on  land.  Geographic  features,  clouds  and  the 
water  surface  provided  ample  visual  contrast.  The  HMD  model  we  used  was  a LEEP  Cyberface  II.  The  LEEP  has  LCD 
displays  479  pixels  wide  by  234  high  and  uses  optics  which  preferentially  magnify  the  periphery  of  each  LCD  image. 
We  used  a psychophysical  technique  to  determine  that  the  LEEP  FOV  is  128°  wide  by  74°  high,  with  30°  of  binocular 
overlap.  We  calculated  an  angular  resolution  per  pixel  of  9.8  arcmin  in  the  center  graduated  to  19.5  arcmin  at  the  edges. 
In  all  our  experiments,  the  rate  of  computing  video  frames  was  30  Hz.  The  devices  we  used  to  track  HMD  position  were 
a Polhemus  3SPACE  FASTRAK  magnetic  device  with  a 19.2K  baud  serial  interface  or  a custom-made  mechanical 
device  connected  directly  to  the  VME  bus  via  an  A/D  board.  We  measured  the  end-to-end  visual  update  delay  of  the 
system  with  both  tracking  devices.  A PC  computer  recorded  signals  from  photometric  devices  picking  up  activity 
directly  from  the  HMD  display  units  and  angular  rate  sensors  on  the  HMD  shell,  while  the  VE  was  running  on  the  SGI. 
The  minimum  delay  we  could  achieve  between  onset  of  HMD  movement  and  image  movement  on  the  HMD  displays 
was  67  ms  with  the  Polhemus  and  21  ms  with  the  mechanical  device.  We  purposely  added  delays  to  the  VE  with  a 
software  ring  buffer  in  order  to  achieve  an  adequate  range  of  experimental  visual  update  delays. 

EXPERIMENTAL  PARADIGM 

Two  separate  experiments  were  conducted.  The  first  experiment  assessed  motion  sickness  severity  at  visual  update 
delays  of  67,  159,  254  and  355  ms  utilizing  the  Polhemus  tracking  device.  Twenty  one  subjects  participated  in  all 
conditions.  Seven  new  subjects  were  recruited  for  the  second  experiment  in  which  motion  sickness  severity  was  assessed 
at  update  delays  of  2 1 , 39,  80  and  163  ms,  achieved  with  the  mechanical  tracker. 

Experimental  exposure  to  the  VE  lasted  15  minutes,  split  into  five  2 minute  sequences  of  head  movements  divided  by  1 
minute  periods  for  rest  and  recording  of  motion  sickness  symptoms.  In  the  2 minute  sequences,  a pre-recorded  audio 
tape  announced  landmarks  in  the  VE  for  the  subject  to  look  at  every  five  seconds  (24  total  movements).  The  required 
head  movements  ranged  from  12°  to  180°  amplitude  in  the  horizontal  plane,  between  25°  up  and  15°  down  in  pitch. 
Subjects  had  been  shown  an  aerial  view  of  the  virtual  harbor  before  donning  the  HMD  in  order  to  familiarize  them  with 
the  landmarks.  The  subjects  stood  while  doing  the  task  and  most  of  the  HMD’s  weight  was  supported  by  long  elastic 
cords.  Typically,  subjects  took  less  than  one  second  to  turn  to  a new  target,  using  a combination  of  eye,  head  and  torso 
rotation  and  sometimes  shifting  their  feet,  and  then  they  stood  still  until  the  next  landmark  was  called. 

Acute  symptoms  of  motion  sickness  were  evaluated  according  to  the  criteria  of  Graybiel,  Miller,  Wood  & Cramer 
(1968).  The  checklist  derived  from  this  scale  measures  five  cardinal  signs  and  symptoms  of  motion  sickness,  including 
nausea,  pallor,  sweating,  salivation  and  drowsiness,  and  additional  qualifying  symptoms  such  as  headache,  dizziness  and 
eye  strain.  Scores  in  the  8-15  point  range  indicate  severe  malaise.  The  checklist  was  filled  out  after  each  two  minute 
sequence  of  head  movements  in  the  VE  and  1 5 minutes  after  leaving  the  VE. 

MOTION  SICKNESS  SIDE  EFFECTS  IN  THE  VE 

Figure  1 illustrates  the  most  severe  motion  sickness  recorded  during  the  15  minute  VE  exposure  period,  for  both 
experiments.  The  severity  of  symptoms  increases  monotonically  as  a function  of  the  visual  update  delay.  Analysis  of 
variance  indicated  significant  effects  of  visual  update  delay:  F(3,  60)=  10.43,  p<.0001  in  Experiment  1 (67,  159,  254,  355 
ms  delays)  and  F(3,  18)=  7.11,  p<  0067  in  Experiment  2 (21,  39,  80,  163  ms  delays).  The  severity  of  symptoms  is 
significantly  greater  than  zero  at  every  delay  except  19  ms  (p<.03  at  least,  individual  t tests).  The  ratings  register  from 
mild  malaise  on  the  Graybiel  scale  at  39  ms  to  moderately  severe  malaise  at  355  ms.  The  severity  of  motion  sickness 
seems  to  asymptote  between  254  and  355  ms  (no  significant  difference  between  these  two).  In  the  254  ms  delay 
condition,  6 of  21  of  subjects  (28%)  refused  to  complete  the  full  15  minute  exposure  because  they  were  too  close  to 
vomiting.  The  full  spectrum  of  symptoms  was  reported  in  all  conditions. 
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1 5 min  HM  + 15min  Rest  + 2 min  HM  + 2 min  HM 

w/HMD  w/o  HMD  w/o  HMD 


Figure  1.  Motion  sickness  elicited  by  voluntary  head  movements  (HM)  in  the  initial  15  minute  exposure  to  a VE  with  a 
254  ms  delay  in  update  of  the  HMD  image,  after  15  minutes  of  rest  w ithout  the  HMD,  2 minutes  of  HM  in  a natural 
environment  and  2 minutes  in  the  VE  again. 

| Pdhemus  Tracker,  N=21 


Figure  2.  Motion  sickness  severity  as  a function  of  visual  update  delay,  in  VEs  created  with  two  different  tracking 
devices.  The  plot  shows  the  maximum  severity  rating  evoked  during  15  minutes  of  intermittent  voluntary  head 
movements. 

MOTION  SICKNESS  AFTEREFFECTS 

A preliminary  analysis  of  the  first  8 subjects  indicated  that  motion  sickness  symptoms  had  abated  almost  entirely  in  15 
minutes  after  the  VE  exposure  period  was  over  and  the  subjects  had  been  resting  quietly.  Therefore,  we  asked  the 
remaining  subjects  after  the  1 5 minute  rest  period  to  perform  two  additional  head  movement  sequences,  guided  by  the 
same  2 minute  tape  recorded  directions  as  before. 

In  the  first  post-rest  sequence,  the  HMD  was  not  worn  and  the  location  of  VE  landmarks  was  marked  by  tags  placed  on 
corresponding  locations  in  the  real  laboratory  environment.  As  soon  as  (he  motion  sickness  symptoms  were  recorded,  the 
subjects  donned  the  HMD  again  and  made  another  2 minute  sequence  of  head  movements  in  the  VE  and  reported 
symptoms  a final  time.  The  extra  conditions  were  only  done  after  the  254  ms  delay  VE  exposure.  Figure  2 illustrates  the 
complete  pattern  of  results  for  the  13  subjects  who  completed  all  conditions. 

The  motion  sickness  severity  experienced  by  this  sub-sample  during  15  minutes  in  the  VE  was  5.7  points,  which  is 
similar  to  the  value  of  6.2  for  the  whole  sample.  Fifteen  minutes  after  leaving  the  VE  and  resting  in  a normal 
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environment  the  malaise  level  was  only  1.0,  not  significantly  above  zero.  Subsequently  performing  head  movements  in  a 
natural  environment  for  2 minutes  brought  the  motion  sickness  score  back  up  to  4.4  points,  which  was  significantly 
above  1.0.  Returning  to  the  VE  and  making  head  movements  for  2 minutes  brought  the  symptom  level  up  above  the 
level  of  the  original  fifteen  minute  exposure,  to  6.7points. 

CONCLUSIONS 

The  results  indicate  that  motion  sickness  is  a serious  potential  side  effect  and  aftereffect  of  using  a HMD  as  a visual 
interface  to  interact  with  a VE.  Motion  sickness  severe  enough  to  convince  subjects  they  might  vomit  can  be  elicited 
within  minutes  of  exposure  to  the  VE.  Users  experience  no  motion  sickness  unless  they  move  their  head  to  scan  the 
environment.  In  the  VEs  we  studied,  there  was  no  simulated  motion  of  the  user’s  base  of  support  or  of  other  objects. 
The  only  visual  motion  was  generated  by  voluntary  head  movements.  Thus,  head  movements  are  a sufficient  cause  of 
motion  sickness  in  VEs  incorporating  HMDs.  This  is  not  the  case  in  flight  simulator. 

Overt  motion  sickness  decays  quickly  after  a 1 5-minute  VE  exposure  is  over,  but  users  remains  sensitized.  Anecdotal 
reports  indicate  the  period  of  sensitization  is  much  longer  than  the  15-minute  period  in  which  we  have  done  formal 
evaluations;  up  to  hours  is  possible.  During  the  sensitized  period,  even  normal  activities  can  bring  the  symptoms  back, 
and  returning  to  the  VE  escalates  symptoms  immediately  to  the  prior  maximum  level  or  above. 

Lags  between  head  movements  and  compensation  of  the  image  in  the  HMD  are  a powerful  etiological  factor  in  this  form 
of  motion  sickness.  Such  lags  distort  visual  motion  of  the  virtual  world  during  head  movements.  The  motion  is  distorted 
in  the  sense  that  the  normal  coordination  of  the  vestibulo-ocular  and  optokinetic  reflexes  does  not  result  in  stabilization 
of  the  retinal  image.  The  magnitude  of  aberrant  visual  motion  and  the  severity  of  motion  sickness  both  increase  in 
proportion  to  the  visual  update  latency.  A latency  of  19  ms  is  below  threshold  for  eliciting  motion  sickness,  under  the 
conditions  we  tested. 

These  findings  can  aid  in  ameliorating  or  circumventing  motion  sickness  in  VEs.  They  suggest  that  careful 
measurements  of  end-to-end  visual  update  delay  of  VE  systems  is  necessary  to  anticipate  and  circumvent  side  effects. 
To  avoid  motion  sickness,  users  can  limit  their  head  movements,  VE  designers  and  engineers  can  strive  for  sub-threshold 
visual  update  delays,  or  the  power  of  human  sensorimotor  adaptation  can  be  exploited. 

We  are  currently  attempting  to  define  other  etiological  factors  and  side  effects  that  must  be  lake  into  consideration.  Our 
results  indicate  that  postural  instability  is  another  serious  side  effect/aftereffect  and  that  the  field  of  view,  weight  and 
spatial  resolution  of  an  HMD  as  well  as  the  computation  rate  of  video  frames  are  other  etiological  factors. 
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